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Abstract — The distributed source coding problem is considered 
when the sensors, or encoders, are under Byzantine attaclt; 
that is, an unknown group of sensors have been reprogrammed 
by a malicious intruder to undermine the reconstruction at 
the fusion center. Three different forms of the problem are 
considered. The first is a variable-rate setup, in which the decoder 
adaptively chooses the rates at which the sensors transmit. 
An explicit characterization of the variable-rate achievable sum 
rates is given for any number of sensors and any groups of 
traitors. The converse is proved constructively by letting the 
traitors simulate a fake distribution and report the generated 
values as the true ones. This fake distribution is chosen so that 
the decoder cannot determine which sensors are traitors while 
maximizing the required rate to decode every value. Achievability 
is proved using a scheme in which the decoder receives small 
packets of information from a sensor until its message can be 
decoded, before moving on to the next sensor. The sensors use 
randomization to choose from a set of coding functions, which 
makes it probabilistically impossible for the traitors to cause 
the decoder to make an error. Two forms of the fixed-rate 
problem are considered, one with deterministic coding and one 
with randomized coding. The achievable rate regions are given 
for both these problems, and it is shown that lower rates can be 
achieved with randomized coding. 

Index Terms — Distributed Source Coding. Byzantine Attack. 
Sensor Fusion. Network Security. 



I. Introduction 

WE consider a modification to the distributed source 
coding problem in which an unknown subset of sensors 
are taken over by a malicious intruder and reprogrammed. 
We assume there are m sensors. Each time slot, sensors i 
for i — 1 , • • ■ , m observe random variables Xi according 
to the joint probability distribution p{xi ■ ■ ■ Xm). Each sensor 
encodes its observation independently and transmits a message 
to a common decoder, which attempts to reconstruct the 
source values with small probability of error based on those 
messages. A subset of sensors are traitors, while the rest are 
honest. Unbeknownst to the honest sensors or the decoder, the 
traitors have been reprogrammed to cooperate to obstruct the 
goal of the network, launching a so-called Byzantine attack. 
To counter this attack, the honest sensors and decoder must 
employ strategies so that the decoder can correctly reconstruct 
source values no matter what the traitors do. 

It is obvious that observations made by the traitors are 
irretrievable unless the traitors choose to deliver them to the 



decoder Thus the best the decoder can hope to achieve is to 
reconstruct the observations of the honest sensors. A simple 
procedure is to ignore the statistical correlations among the 
observations and collect data from each sensor individually. 
The total sum rate of such an approach is J2i ^i-^i)- 
expects however that this sum rate can be lowered if the 
correlation structure is not ignored. 

Without traitors, Slepian-Wolf coding [1] can be used to 
achieve a sum rate as low as 



H{Xi ■ ■ ■ X„j). 



(1) 



However, standard Slepian-Wolf coding has no mechanism for 
handling any deviations from the agreed-upon encoding func- 
tions by the sensors. Even a random fault by a single sensor 
could have devastating consequences for the accuracy of the 
source estimates produced at the decoder, to say nothing of 
a Byzantine attack on multiple sensors. In particular, because 
Slepian-Wolf coding takes advantage of the correlation among 
sources, manipulating the codeword for one source can alter 
the accuracy of the decoder's estimate for other sources. It will 
turn out that for most source distributions, the sum rate given 
in (HJ cannot be achieved if there is even a single traitor 

In this paper, we are interested in the lowest achievable sum- 
rate such that the decoder can reconstruct observations of the 
honest sensors with arbitrarily small error probability. In some 
cases, we are also interested in the rate region. We note that 
although the problem setup does not allow the detector to dis- 
tinguish traitors from the honest sensors, an efficient scheme 
that guarantees the reconstruction of data from honest sensors 
is of both theoretical and practical interest. For example, for 
a distributed inference problem in the presence of Byzantine 
sensors, a practical (though not necessarily optimal) solution is 
to attack the problem in two separate phases . In the first phase, 
the decoder collects data from sensors over multiple access 
channels with rate constraints. Here we require that data from 
honest sensors are perfectly reconstructed at the decoder even 
though the decoder does not know which piece of data is from 
an honest sensor In the second step, the received data is used 
for statistical inference. The example of distributed detection 
in the presence of Byzantine sensors is considered in [2]. 
The decoder may also have other side information about the 
content of the messages that allows the decoder to distinguish 
messages from the honest sensors. 



This work is supported in part by the National Science Foundation 
under award CCF-0635070, the U. S. Army Research Laboratory under the 
Collaborative Technology Alhance Program DAAD19-01-2-001 1, and TRUST 
(The Team for Research in Ubiquitous Secure Technology) sponsored by the 
National Science Foundation under award CCF-0424422. 



A. Related Work 

The notion of Byzantine attack has its root in the Byzantine 
generals problem [3], [4] in which a clique of traitorous 
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generals conspire to prevent loyal generals from forming 
consensus. It was shown in [3] that consensus in the presence 
of Byzantine attack is possible if and only if less than a third 
of the generals are traitors. 

Countering Byzantine attacks in communication networks 
has also been studied in the past by many authors. See the 
earlier work of Perlman [5] and also more recent review 
[6], [7]. An information theoretic network coding approach 
to Byzantine attack is presented in [8J. In [9J, Awerbuch 
et al suggest a method for mitigating Byzantine attacks on 
routing in ad hoc networks. Their approach is most similar 
to ours in the way ihey maintain a list of current knowledge 
about which links are trustworthy, constantly updated based 
on new information. Sensor fusion with Byzantine sensors 
was studied in [10]. In that paper, the sensors, having already 
agreed upon a message, communicate it to the fusion center 
over a discrete memoryless channel. Quite similar results were 
shown in [11], in which a maUcious intruder takes control 
of a set of links in the network. The authors show that two 
nodes can conmiunicate at a nonzero rate as long as less 
than half of the links between them are Byzantine. This is 
different from the current paper in that the transmitter chooses 
its messages, instead of relaying information received from an 
outside source, but some of the same approaches from [11] are 
used in the current paper, particularly the use of randomization 
to fool traitors that have already transmitted. 

B. Redefining Achievable Rate 

The nature of Byzantine attack require three modifications 
to the usual notion of achievable rate. The first, as mentioned 
above, is that small probability of error is required only for 
honest sources, even though the decoder may not know which 
sources are honest. This requirement is reminiscent of [3], 
in which the heutenants need only perform the commander's 
order if the commander is not a traitor, even though the 
lieutenants might not be able to decide this with certainty. 

The next modification is that there must be small probability 
of error no matter what the traitors do. This is essentially the 
definition of Byzantine attack. 

The final modification has to do with which sensors are 
allowed to be traitors. Let 'K be the set of honest sensors, and 
T = {1, ■ • • , m}\J{ the set of traitors. Any code is associated 
with a list of which sets of sensors it can handle as the set 
of traitors. A rate is then achieved if the code gets small 
probability of error when the actual set of traitors is in fact on 
the list. It will be more convenient to specify not the list of 
allowable sets of traitors, but rather the list of allowable sets of 
honest sensors. We define ,^ C 2'^^^- '™} to be this fist. Thus 
small probability of error is required only when "K G M'. One 
special case is when the code can handle any group of at most 
t traitors. That is, 

M' = Mi = {$c{l,--- ,m}:\$\>m-t}. 

Observe that achievable rates depend not just on the true set 
of traitors but also on the collection M', because the decoder's 
wilUngness to accept more and more different groups of 
traitors allows the true traitors to get away with more without 



being detected. Thus we see a trade off between rate and 
security — ^in order to handle more traitors, one needs to be 
wUUng to accept a higher rate. 

C. Fixed-Rate Versus Variable-Rate Coding 

In standard source coding, an encoder is made up of a 
single encoding function. We will show that this fixed-rate 
setup is suboptimal for this problem, in the sense that we 
can achieve lower sum rates using variable-rate coding. By 
variable-rate we mean that the number of bits transmitted 
per source value by a particular sensor will not be fixed. 
Instead, the decoder chooses the rates at "run time" in the 
following way. Each sensor has a finite number of encoding 
functions, all of them fixed beforehand, but with potentially 
different output alphabets. The coding session is then made up 
of a number of transactions. Each transaction begins with the 
decoder deciding which sensor will transmit, and which of its 
several encoding functions it will use. The sensor then executes 
the chosen encoding function and transmits the output back to 
the decoder. Finally, the decoder uses the received message to 
choose the next sensor and encoding function, beginning the 
next transaction, and so on. Thus a code is made up of a set of 
encoding functions for each sensor, a method for the decoder 
to choose sensors and encoding functions based on previously 
received messages, and lastly a decoding function that takes 
all received messages and produces source estimates. 

Note that the decoder has the ability to transmit some 
information back to the sensors, but this feedback is limited to 
the choice of encoding function. Since the number of encoding 
functions need not grow with the block length, this represents 
zero rate feedback. 

In variable-rate coding, since the rates are only decided 
upon during the coding session, there is no notion of an m- 
dimensional achievable rate region. Instead, we only discuss 
achievable sum rates. 

D. Traitor Capabilities 

An important consideration with Byzantine attack is the 
information to which the traitors have access. First, we assume 
that the traitors have complete knowledge of the coding 
scheme used by the decoder and honest sensors. Furthermore, 
we always assume that they can communicate with each 
other arbitrarily. For variable-rate coding, they may have any 
amount of ability to eavesdrop on transmissions between 
honest sensors and the decoder. We will show that this ability 
has no effect on achievable rates. We assume with fixed-rate 
coding that all sensors transmit simultaneously, so it does not 
make sense that traitors could eavesdrop on honest sensors' 
transmissions before making their own, as that would violate 
causality. Thus we assume for fixed-rate coding that the traitors 
cannot eavesdrop. 

The key factor, however, is the extent to which the traitors 
have direct access to information about the sources. We assume 
the most general memoryless case, that the traitors have access 
to the random variable W, where W is i.i.d. distributed 
with {X\ ■ ■ ■ Xm) according to the conditional distribution 
r{w\xi ■ ■ ■ Xm)- A natural assumption would be that W always 
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includes Xi for traitors i, but in fact this need not be the case. 
An important special case is where W = {Xi, ■ ■ ■ ,Xm), i-S- 
the traitors have perfect information. 

We assume that the distribution of W depends on who 
the traitors are, and that the decoder may not know exactly 
what this distribution is. Thus each code is associated with a 
function that maps elements of to sets of conditional 
distributions r. The relationship between r and 3l(Jf) is 
analogous to the relationship between J{ and J^. That is, given 
!K, the code is willing to accept all distributions r G 3l(!K). 
Therefore a code is designed based on and % and then 
the achieved rate depends at run time on 3i and r, where 
we assume 3i G J^f and r G We therefore discuss 

not achievable rates R but rather achievable rate functions 
R{'K, r). In fact, this applies only to variable-rate codes. In 
the fixed-rate case, no run time rate decisions can be made, 
so achievable rates depend only on and 51. 



E. Main Results 

The main results of this paper give explicit characterizations 
of the achievable rates for three different setups. The first, 
which is discussed in the most depth, is the variable-rate 
case, for which we characterize achievable sum rate functions. 
The other two setups are for fixed-rate coding, divided into 
deterministic and randomized coding, for which we give Tri- 
dimensional achievable rate regions. We show that randomized 
coding yields a larger achievable rate region than deterministic 
coding, but we believe that in most cases randomized fixed- 
rate coding requires an unrealistic assumption. In addition, 
even randomized fixed-rate coding cannot achieve the same 
sum rates as variable-rate coding. 

We give the exact solutions in Theorems 1 and 2, but 
describe here the intuition behind them. For variable-rate, 
the achievable rates are based on alternate distributions on 
{Xi ■ ■ ■ Xm). Specifically, given W, the traitors can simulate 
any distribution q{x'r\w) to produce a fraudulent version of 
X!^, then report this sequence as the truth. Suppose that the 
overall distribution q{xi ■ ■ ■ x„j) governing the combination 
of the true value of XJ^ with this fake value of X]^ could 
be produced in several different ways, with several different 
sets of traitors. In that case, the decoder cannot tell which 
of these several possibilities is the truth, which means that 
from its point of view, any sensor that is honest in one 
of these possibilities may in fact be honest. Since the error 
requirement described in II-BI stipulates that the decoder must 
produce a correct estimate for every honest sensor, it must 
attempt to decode the source values associated with all these 
potentially honest sensors. Thus the sum rate must be at least 
the joint entropy, when distributed according to q, of the 
sources associated with all potentially honest sensors. The 
supremum over all such qs is the achievable sum rate. 

For example, suppose = J^m-i- That is, at most one 
sensor is honest. Then the traitors are able to create the 
distribution q{xi ■ ■ ■ x„i) — p{xi) ■ ■ ■p{xm) no matter what 
group of TO — 1 sensors are the traitors. Thus every sensor 
appears as if it could be the honest one, so the minimum 



achievable sum rate is 

H{X^) + --- + H{X„,). (2) 

In other words, the decoder must use an independent source 
code for each sensor, which requires receiving nH{Xi) bits 
from sensor i for all i. 

The achievable fixed-rate regions are based on the Slepian- 
Wolf achievable rate region. For randomized fixed-rate coding, 
the achievable region is such that for all § G ,y«f, the rates 
associated with the sensors in § fall into the Slepian-Wolf 
rate region on the corresponding random variables. Note that 
for — {{1, • • • ,to}}, this is identical to the Slepian-Wolf 
region. For = ,j^„i-i, this region is such that for all i, 
Ri > H{Xi), which corresponds to the sum rate in (|2]l. The 
deterministic fixed-rate achievable region is a subset of that of 
randomized fixed-rate, but with an additional constraint stated 
in Section |VI] 

F. Randomization 

Randomization plays a key role in defeating Byzantine 
attacks. As we have discussed, allowing randomized encoding 
in the fixed-rate situation expands the achievable region. In ad- 
dition, the variable-rate coding scheme that we propose relies 
heavily on randomization to achieve small probability of error 
In both fixed and variable-rate coding, randomization is used 
as follows. Every time a sensor transmits, it randomly chooses 
from a group of essentially identical encoding functions. The 
index of the chosen function is transmitted to the decoder 
along with its output. Without this randomization, a traitor 
that transmits before an honest sensor i would know exactly 
the messages that sensor i will send. In particular, it would be 
able to find fake sequences for sensor i that would produce 
those same messages. If the traitor tailors the messages it 
sends to the decoder to match one of those fake sequences, 
when sensor i then transmits, it would appear to corroborate 
this fake sequence, causing an error. By randomizing the 
choice of encoding function, the set of sequences producing 
the same message is not fixed, so a traitor can no longer 
know with certainty that a particular fake source sequence 
will result in the same messages by sensor i as the true 
one. This is not unlike Wyner's wiretap channel [12], in 
which information is kept from the wiretapper by introducing 
additional randomness. See in particular Section IV-DI for the 
proof that variable-rate randomness can defeat the traitors in 
this manner. 

The rest of the paper is organized as follows. In Section Ull 
we develop in detail the case that there are three sensors 
and one traitor, describing a coding scheme that achieves 
the optimum sum rate. In Section Hill we formally give the 
variable-rate model and present the variable-rate result. In 
Section HVl we discuss the variable-rate achievable rate region 
and give an analytic formulation for the minimum achievable 
sum rate for some special cases. In Section [Vll we give 
the fixed-rate models and present the fixed-rate result. In 
Sections |V] and IVIII we prove the variable-rate and fixed-rate 
results respectively. Finally, in Section IVIIII we conclude. 



4 



II. Three Sensor Example 

A. Potential Traitor Techniques 

For simplicity and motivation, we first explore the three- 
sensor case with one traitor That is, m — 3 and 

^-{{1,2},{2,3},{1,3}}. 

Suppose also that the traitor has access to perfect information. 
Consider first the simple case where the Xi can be decom- 
posed as 

^1 = (^1,5^12,^13:^123), 
X2 — (5^2,^12,5^23,^123), 
-^^3 = (53, 5^13, 5^23, 5^23) 

where Yi, >2, 51j, 5^2, 5^i3, 5^23, 5^123 are independent. Suppose 
the traitor is sensor 3. It can generate a new, indepen- 
dent version of 123, call it ¥2^, and then form = 
(Yi, Yia, 5^23, 5^123)- We claim that if sensor 3 now behaves for 
the rest of the coding session as if this counterfeit X^ were the 
real value, then the decoder will not be able to determine the 
traitor's identity. This is because both {Xi,X2) and {X2,X^) 
look like they could be a true pair, since all information that 
they share matches. Thus the decoder cannot know which of 
sensors 1 or 3 is the traitor, and which of I23 or is the 
truth, so it must obtain estimates of them both. To construct 
estimates of all three variables, every piece except 123 must be 
received only once, but the two versions >23 must be received 
separately. Therefore the sum rate must be at least 

H{XiX2X3) + HiY23) = H{XiX2X3) + IiX2; X^lX^). 

(3) 

In fact, this last expression holds for general distributions as 
well, as we demonstrate next. 

Now take any distribution p, again with sensor 3 as the 
traitor. Sensors 1 and 2 will behave honestly, so they will 
report Xi and X2 correctly, as distributed according to the 
marginal distribution p(xi 2:2). Since sensor 3 has access to the 
exact values of Xi and X2, it may simulate the conditional 
distribution p{x3\x2), then take the resulting X3 sequence 
and report it as the truth. Effectively, then, the three random 
variables will be distributed according to the distribution 

9(2:1X2X3) =p(xiX2)p(x3|x2). 

The decoder will be able to determine that sensors 1 and 2 are 
reporting jointly typical sequences, as are sensors 2 and 3, but 
not sensors 1 and 3. Therefore, it can tell that either sensor 1 or 
3 is the traitor, but not which one, so it must obtain estimates 
of the sources from all three sensors. Since the three streams 
are not jointly typical with respect to the source distribution 
p{xiX2X3), Standard Slepian-Wolf coding on three encoders 
will not correctly decode them all. However, had we known 
the strategy of the traitor, we could do Slepian-Wolf coding 
with respect to the distribution q. This will take a sum rate of 

Hg{XiX2X3) = H{XiX2X3)+I{Xi;X3\X2) 

where Hq is the entropy with respect to q. In fact we will not 
do Slepian-Wolf coding with respect to q but rather something 
slightly different that gives the same rate. Observe that this 



matches ([3]l. Since Slepian-Wolf coding without traitors can 
achieve a sum rate of H{XiX2X3), we have paid a penalty 
of I{Xi; X3\X2) for the single traitor. 

We supposed that sensor 3 simulated the distribution 
p(x3|x2). It could have just as easily simulated p(x3|xi), 
or another sensor could have been the traitor. Hence, the 
minimum achievable sum rate for all G is at least 

R* ^ HiXiX2X3) + ma.x{IiXi;X2\X3), 

I{Xi;X3\X2),IiX2:X3\Xi)}. (4) 

In fact, this is exactly the minimum achievable sum rate, as 
shown below. 

B. Variable-Rate Coding Scheme 

We now give a variable-rate coding scheme that achieves 
R* . This scheme is somewhat different from the one we 
present for the general case in Section [V] but it is much 
simpler, and it illustrates the basic idea. The procedure will be 
made up of a number of rounds. Communication from sensor 
i in the first round will be based solely on the first n values of 
Xi, in the second round on the second n values of Xi, and so 
on. The principle advantage of the round structure is that the 
decoder may hold onto information that is carried over from 
one round to the next. 

In particular, the decoder maintains a collection Y C Jif 
representing the sets that could be the set of honest sensors. 
If a sensor is completely eliminated from that means it 
has been identified as the traitor. We begin with f = M' , and 
then remove a set from "V whenever we find that the messages 
from the corresponding pair of sensors are not jointly typical. 
With high probability, the two honest sensors report jointly 
typical sequences, so we expect never to eliminate the honest 
pair from 'f . If the traitor employs the q discussed above, for 
example, we would expect sensors 1 and 3 to report atypical 
sequences, so we will drop {1,3} from y . In essence, the 
value of contains our current knowledge about what the 
traitor is doing. 

The procedure for a round is as follows. If contains 
{{1, 2}, {1,3}}, do the following: 

1) Receive nH{Xi) bits from sensor 1 and decode x". 

2) Receive nH (X2\Xi) bits from sensor 2. If there is a 
sequence in jointly typical with x" that matches this 
transmission, decode that sequence to X2 . If not, receive 
nI{Xi; X2) additional bits from sensor 2, decode X2, 
and remove {1,2} from 

3) Do the same with sensor 3: Receive nH {X^lXi) bits 
and decode Xg if possible. If not, receive nI(Xi; X^) 
additional bits, decode, and remove {1,3} from Y. 

If y is one of the other two subsets of J^f with two 
elements, perform the same procedure but replace sensor 1 
with whichever sensor appears in both elements in Y. If Y 
contains just one element, then we have exactly identified the 
traitor, so ignore the sensor that does not appear and simply 
do Slepian-Wolf coding on the two remaining sensors. 

Note that the only cases when the number of bits transmitted 
exceeds nR* are when we receive a second message from one 
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of the sensors, which happens exactly when we eliminate an 
element from . Assuming the source sequences of the two 
honest sensors are jointly typical, this can occur at most twice, 
so we can always achieve a sum rate of R* when averaged 
over enough rounds. 

C. Fixed-Rate Coding Scheme 

In the procedure described above, the number of bits sent 
by a sensor changes from round to round. We can no longer do 
this with fixed-rate coding, so we need a different approach. 
Suppose sensor 3 is the traitor It could perform a black hole 
attack, in which case the estimates for X'1 and must be 
based only on the messages from sensors 1 and 2. Thus, the 
rates R\ and Ri must fall into the Slepian-Wolf achievability 
region for X\ and X2- Similarly, if one of the other sensors 
was the traitor, the other pairs of rates also must fall into the 
corresponding Slepian-Wolf region. Putting these conditions 
together gives 

i?! > max{F(Xi|X2), J?(Xi|X3)} 
i?2 > max{7J(X2|Xi),i?(X2|X3)} 
i?3 > max{i7(X3|Xi), E(X^\X2)} 

Ri+R2> H{XiX2) 

Ri +R3> HiX.Xs) 

R2 + R3>H{X2X3). 

If the rates fall into this region, we can do three simultaneous 
Slepian-Wolf codes, one on each pair of sensors, thereby 
constructing two estimates for each sensor If we randomize 
these codes using the method described in Section II-FI the 
traitor will be forced either to report the true message, or report 
a false message, which with high probability will be detected 
as such. Thus either the two estimates for each sensor will 
be the same, in which case we know both are correct, or one 
of the estimates will be demonstrably false, in which case the 
other is correct. 

We now show that the region given by Q does not include 
sum rates as low as R*. Assume without loss of generality 
that I{Xi; X2\X3) achieves the maximum in (|4]l. Summing 
the last three conditions in ^ gives 

Ri + R2 + R3> ^{HiXiX2) + H{XiX3) + H{X2X3)) 

= H{XiX2X^) + i(/(Xi;X2|X3) +/(XiX2;X3)). (6) 

If I{XiX2;Xz) > I{Xi;X2\X3), © is larger than Q. 
Hence, there exist source distributions for which we cannot 
achieve the same sum rates with even randomized fixed-rate 
coding as with variable-rate coding. 

If we are interested only in deterministic codes, the region 
given by (|5]l can no longer be achieved. In fact, we will prove 
in Section lVTIl that the achievable region reduces to the trivially 
achievable region where Ri > H{Xi) for all i when m = 3, 
though it is nontrivial for m > 3. For example, suppose m = 4 
and J/f — M\. In this case, the achievable region is similar 
to that given by (|5]l, but with an additional sensor. That is, 
each of the 6 pairs of rates must fall into the corresponding 



Slepian-Wolf region. In this case, we do three simultaneous 
Slepian-Wolf codes for each sensor, construct three estimates, 
each associated with one of the other sensors. For an honest 
sensor, only one of the other sensors could be a traitor, so at 
least two of these estimates must be correct. Thus we need 
only take the plurality of the three estimates to obtain the 
correct estimate. 

III. Variable-Rate Model and Result 

A. Notation 

Let Xi be the random variable revealed to sensor i, the 
alphabet of that variable, and Xi a corresponding realization. 
A sequence of random variables revealed to sensor i over n 
timeslots is denoted X", and a realization of it £ X". Let 
M = {1, • • • , -fn). For a set S C M, let Xg be the set of 
random variables {Xi}ig§, and define xg and Xg similarly. 
By S'^ we mean M\§. Let T^(X^)\q\ be the strongly typical 
set with respect to the distribution q, or the source distribution 
•p if unspecified. Similarly, Hq{X$) is the entropy with respect 
to the distribution q, or p if unspecified. 

B. Communication Protocol 

The transmission protocol is composed of L transactions. 
In each transaction, the decoder selects a sensor to receive 
information from and selects which of K encoding functions 
it should use. The sensor then responds by executing that 
encoding function and transmitting its output back to the 
decoder, which then uses the new information to begin the 
next transaction. 

For each sensor i e M and encoding function j G 
{1, ■ • ■ ,K}, there is an associated rate Ri j. On the kh 
transaction, let ii be the sensor and ji the encoding function 
chosen by the decoder, and let hi be the number of /' G 
{1, • • • ,1 — 1} such that ii' — ii. That is, hi is the number 
of times ii has transmitted prior to the Ith transaction. Note 
that ii , ji , hi are random variables, since they are chosen by 
the decoder based on messages it has received, which depend 
on the source values. The jth encoding function for sensor i 
is given by 

: X^ X Z X {1, • • • , K}''' ^ {1, • • • , 2"«- } (7) 

where Z represents randomness generated at the sensor Let 
/; G {I,-- - ,2"'^'i.n} be the message received by the 
decoder in the Ith transaction. If ii is honest, then /; = 
fii,ji{X"^, Pii, Ji), where pi^ G Z is the randomness from 
sensor i; and Ji G {1, ■ • ■ ,K}'^' is the history of encoding 
functions used by sensor ii so far. If ii is a traitor, however, 
it may choose /; based on VF" and it may have any amount 
of access to previous transmissions /i, • • • ,Ii^i and polling 
history zi, • • ■ , ii-i and ji, • ■ • ,ji-i- But, it does not have 
access to the randomness pi for any honest sensor i. Note 
again that the amount of traitor eavesdropping ability has no 
effect on achievable rates. 

After the decoder receives Ii, if I < L it uses /i, • • • , // to 
choose the next sensor ij+i and its encoding function index 
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ji+i- After the Lth transaction, it decodes according to the 
decoding function 

L 

1=1 

Note that we impose no restriction whatsoever on the size 
of the total number of transactions L. Thus, a code could 
have arbitrary complexity in terms of the number of messages 
passed between the sensors and the decoder However, in 
our below definition of achievability, we require that the 
communication rate from sensors to decoder always exceeds 
that from decoder to sensors. Therefore while the number 
of messages may be very large, the amount of feedback is 
dinimishingly small. 

C. Variable-Rate Problem Statement and Main Result 

Let !K C M be the set of honest sensors. Define the 
probability of error 

where (Xf,--- ,X^) ^ .g(/i,-- - The probability of 

error will in general depend on the actions of the traitors. 
Note again that we only require small probability of error on 
the source estimates corresponding to the honest sensors. 

We define a rate function i?(!K, r) defined for "K G Jff and 
r £ 3?(!K) to be a-achievable if there exists a code such that, 
for all pairs (!K, r) and any choice of actions by the traitors, 
Pe < a, 

Pr(^i?.,,,, <R{'K,r) \ >l-a 
^ 1=1 ^ 

and log K < anRij for all This last condition requires, as 
discussed above, that the feedback rate from the decoder back 
to the sensors is arbitrarily small compared to the forward rate. 
A rate function R{3i, r) is achievable if for all a > 0, there 
is a sequence of a-achievable rate functions {^j.(^H, r)}^j^ 
such that 

lim R'k{'K,r) = R{'K,r). 

k — 'oc 

Note that we do not require uniform convergence. 

The following definitions allow us to state our main 
variable-rate result. For any 5{ G ^ and r e 3?(5{), let 

f{w\x^)= ^ p{x:H'\x^)r{w\x^x^a). 

The extent to which W provides information about Xj^o is 
irrelevant to the traitors, since all that really matters to the 
traitors is generating information that appears to agree with 
Xj^ as reported by the honest sensors. Thus it will usually be 
more convenient to work with f rather than r. For any S £ 
and r' e 3^(8), let 

Qs,r' = < p{x8)'^r' {w\x§)q{x§a\w) : Wq{x8a\w) L 

If were the traitors and W were distributed according to r', 
Qs.r' is the set of distributions q to which the traitors would 



have access. That is, if they simulate the proper q{x$c \w) from 
their received W and combine the result with the actual value 
of x§, the combination is distributed according to q. For any 
r C define 

Q(r) ^ n U Qs,.'. 

Ser -r'e3J(S) 

That is, for some distribution q G Q(;^), for every § e Y, 
if the traitors were S"^, they would have access to q for some 
r' G 3l(S). Thus any distribution in Q{'f) makes it look to the 
decoder like any S G 1^ could be the set of honest sensors, so 
any sensor in 11(7^) = Use t-' ^ potentially honest. 

Theorem 1: A rate function r) is achievable if and 

only if, for all (?{,r), 

i?(5{,r) > i?*(5{,r) ^ sup H,(Xu(y)). 

(8) 

See Section |V] for the proof. 

IV. Properties of the Variable-Rate Region 

It might at first appear that (|9]l does not agree with (|4|l. 
We discuss several ways in which (O and (|9]l can be made 
more manageable, particularly in the case of perfect traitor 
information, and show that the two are in fact identical. Let 
R* be the minimum rate achievable over all "K G M' and 
r G 3^(3i). Thus by we can write 

R* ^ sup i?*(5f,r)= sup Hq{Xu(^)). 

(9) 

This is the quantity that appears in (|4]l. Note also that for 
perfect traitor information, 

Qs,r' = {q{xM) ■ q{xs) = p{xs)}. (10) 

This means that Qjr,^ n Q(r) = Q(r U {?{}). Therefore ^ 
becomes 

i?*(J{,r)- sup HgiXuir)). 

The following lemma simplifies calculation of expressions of 
the form supggQ(-^) Hq{Xu(-r)). 

Lemma 1: Suppose the traitors have perfect information. 
For any Y C ,ytf, the expression 

sup HgiXu(r)) (11) 

is maximized by a q satisfying dTol l for all 8 G 1^ such that, 
for some set of functions {a$}$^'r, 

• • -a;™) crs(a;s)- (12) 

Proof: By ( fTOl i. we need to maximize Hq{Xii{'r)) subject 
to the constraints that for each § G Y and all x$ G Xg, 
q{xs) = p{xs)- This amounts to maximizing the Lagrangian 

A = - ^ qixui-r) ) log qixui-r) ) 

+ X! X! ^s{x$){q{xs) ~ p{x$)). 

serxgeXg 
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Note that for any § C 

dq{xs) 



^ 1. 



Thus, differentiating with respect to q{xu{'r)) gives, assuming 
the log is a natural logarithm. 



dA 



dq{xucr)) 
Setting this to gives 



logg(a;u(r)) - 1 + ^ As(a;s). 



9(a;u(r)) = exp f -1+^ As(xs) j = |Xu(r)-l H ^^§(2:8) 

for some set of functions {(Ts}se r- Therefore setting 

q{xu{r)) 



q{xi ■ ■ ■ Xrn) 



satisfies ST% . so if as are such that ( fTOl i is satisfied for all 
§ ^ i^, q will maximize Hq{Xin-y^). ■ 
Suppose m = 3 and = ^1. If ^ = {{1, 2}, {2, 3}}, 
then q(xia;2a;3) = p{xiX2)p{x3\x2) is in 0(1^) and by 
Lemma [T] maximizes Hq(XiX2X^) over all q G 0(1^). Thus 

sup HqiX^X^X:,) = i?,-(^1^2^3) 

<?eQ(r) 

= i/(XiX2X3) + /(Xi;X3|X2). 

By similar reasoning, considering Y — {{1, 2}, {1, 3}} and 
y = {{1, 3}, {2, 3}} results in (H. Note that if Yi C ^2, then 
0(1^1) D Q('3^), so 1^ need not be considered in evaluating 
dSJl. Thus we have ignored larger subsets of J^, since the 
value they give would be no greater than the others. 

We can generalize to any collection Y of the form 
{{§1, §2}, {§1, §3}, • • • , {§1, Sfc}}, in which case 

sup = H{Xs,Xs,) + H{Xs, |Xs J + ■ • • + i/(Xs, |Xs J. 

Employing this, we can rewrite (|9]l for ^ ~ Mt and certain 
values of t. For t = 1, it becomes 



TC — H{X\ ■ ■ ■ X„ 



- max I{Xi;Xi>\Xuinc). 



Again, relative to the Slepian-Wolf result, we always pay a 
conditional mutual information penalty for a single traitor. For 

t = 2, 

R* — H{Xi ■ ■ ■ Xjn) 

+ max<; max I{Xs; X$'\X/§^j§nc), 

S,S'CM:|S| = |S'|=2 < \ ) ' 



max I{Xi; Xi'] Xi',\Xu i, i„\a) > 



where I{X; Y; Z\W) = H{X\W) + H{Y\W) + H{Z\W) - 
H{XYZ\W). For i = TO - 1, R* is given by ©. There is a 
similar formulation for t — rn — 2, though it is more difficult 
to write down for arbitrary to. 

With all these expressions made up of nothing but entropies 
and mutual informations, it might seem hopeful that (fTTT i 
can be reduced to such an analytic expression for all Y. 
However, this is not the case. For example, consider Y — 



{{1,2, 3}, {3, 4, 5}, {5, 6,1}}. This r is irreducible in the 
sense that there is no subset Y' that still satisfies "{[{Y') = 
{1, • • • , 6}, but there is no simple distribution q G Q{Y) made 
up of marginals of p that satisfies Lemma [T] so it must be 
found numerically. Still, Lemma [T] simplifies the calculation 
considerably. 



V. Proof of Theorem [T| 



A. Converse 



We first show the converse. Fix J{ G J^f and r G 3?(5{). 
Take any Y C Jf, and any distribution q G Qjt,r n Q{Y). 
Since q G Q^K.r, there is some ([{xrlw) such that X^ and 
X-j are distributed according to q. Since also q G Qs.r' for all 
§ ^ Y and some r' G 3?(§), if the traitors simulate this q and 
act honestly with these fabricated source values, the decoder 
will not be able to determine which of the sets in Y is the 
actual set of honest sensors. Thus, the decoder must perfectly 
decode the sources from all sensors in 'U.{Y), so if R{'K, r) is a 
precisely a-achievable rate function, R{'K,r) > Hq{Xii(^y^). 

B. Achievability Preliminaries 

Now we prove achievability. To do so, we will first need 
the theory of types. Given G y", let be the type 

of y". Given a type t with denominator n, let A"(F) be the 
set of all sequences in y" with type t. If t is a joint y, z 
type with denominator n, then let A"(y|z") be the set of 
sequences y" G y " such that have joint type t, with 

the convention that this set is empty if the type of z" is not 
the marginal of t. 

We will also need the following definitions. Given a distri- 
bution q on an alphabet y, define the ?7-ball of distributions 

B,(g)^|g'(y):V:rGy:k(x)-9'(x)|<^|. 

Note that the typical set can be written 

T^{X) = {a;" : G B^)- 

We define slightly modified versions of the sets of distributions 
from Section UlI-CI as follows: 

U B,{q\ 

Q^[Y) ^ n u K'- 

Finally, we will need the following lemma. 

Lemma 2: Given an arbitrary n length distribution (/"(a;") 
and a type t with denominator n on X, let qi (x) be the marginal 
distribution of at time i and q{x) = - 'Y^^=i Qii^)- If 
is distributed according to q" and Pr(A:"G AJ'(X)) > 2-'^, 
then D{t\\q) < C 

Proof: Fix an integer n. For i = 1, • • • , ri, let A'"(i) 
be independently generated from g". Let F be the set of 
types on supersymbols in X" with denominator n such 
that r(a;") = if a;" ^ A'^iX). Note that 



|r| < (n + 1) 



xr 



g 



If X"" = (X"(l), • ■ • , then 

Pr (x"" e y A[i.(X")) = Pr(X"(i) e A^{x)yi) 

But 

Pr (x™ e y Af„(X")) = ^ Pr(X"" € A%{X") 



t'>Gr 
< (n + 1) 



|X|' 



-nmintngr -D(t"||g") 



For any i" G F, letting ti be the marginal type at time i gives 
^ I]r=i ^« = Therefore 

( + J_|X|"log(n + l) > min-D(r||g") 



1 " 

> imn -Y D{U\\q,) (13) 



> Dm) 



(14) 



where (fT3T l holds by [13, Lemma 4.3] and ( fT4l l by convexity 
of the Kullback-Leibler distance in both arguments. Letting n 
grow proves the lemma. ■ 
The achievability proof proceeds as follows. Section [V-CI 
describes our proposed coding scheme for the case that traitors 
cannot eavesdrop. In Section IV-DI we demonstrate that this 
coding scheme achieves small probability of error when the 
traitors have perfect information. Section IV-EI shows that 
the coding scheme achieves the rate function R*{'K,r). In 
Section IV-FI we extend the proof to include the case that 
the traitors have imperfect information. Finally, Section IV-GI 
gives a modification to the coding scheme that can handle 
eavesdropping traitors. 

C. Coding Scheme Procedure 

1) Random Code Structure: Fix e > 0. The codebook for 
sensor i is composed of CJi separate encoding functions. 



where Ji — 
In particular 



e 

for z = 1 , • • • , m and c — 1 
fi,c,j ■ 3Cj > {1, • • • , 2 }, j — 2, 



and C is an integer to be defined later 
, C, let 



1 J i 



with ly to defined later We put tildes on these functions to 
distinguish them from the /s defined in (|7]). The /s that 
we define here are functions we use as pieces of the overall 
encoding functions /. Each one is constructed by a uniform 
random binning procedure. For a given i and c, one can think 
of {fi,c,j}j as a subcodebook that associates each S X" 
with a long sequence of bits split into blocks of length nle+iy) 
or ne. Define composite functions 

We can think of Fi c j(a;") as an index of one of 2"'^-''^+'') 
random bins. 



2) Round Method: We propose a coding scheme made up 
of N rounds, with each round composed of m phases. In the 
ith phase, transactions are made entirely with sensor i. We 
denote as the /th block of n source values, but for 
convenience, we will not include the index / when it is clear 
from context. As in the three-sensor example, all transactions 
in the /th round are based only on X^{I). Thus the total 
block length is Nn. 

The procedure for each round is identical except for the 
variable maintained by the decoder. This represents the 

collection of sets that could be the set of honest sensors based 
on information the decoder has received as of the beginning 
of round /. The decoder begins by setting ^(1) = and 
then pares it down at the end of each round based on new 
information. 

3) Encoding and Decoding Rules: In the ith phase, if i G 

the decoder makes a number of transactions with 
sensor i and produces an estimate X" of X". The estimate 
X" is of course a random variable, so as usual the lower case 
refers to a realization of this variable. If i ^ U{'f{I)), then 
the decoder has determined that sensor i cannot be honest, so 
it does not communicate with it and sets xf to a null value. 

For i £ at the beginning of phase i, sensor i 

randomly selects a c G {1, ■ ■ ■ i C"}- In th^ first transaction, 
sensor i transmits (c, /i_c,i(A'")). As the phase continues, in 
the jth transaction, sensor i transmits fi^cj{Xf). 

After each transaction, the decoder decides whether to ask 
for another transaction based on the following rubric. For any 
s C M and e X" let 



Note that 



Let Si ^ {I,-- - ,i} r\\i{y) and be the previously 

decoded source sequences in this round. After j transactions, 
the decoder will choose to do another transaction if there are 
no sequences in Tj{xsi_.^) rnatching the received value of 
Fi c.j- If there is at least one such sequence, let a;" be one 
such sequence. If there are several, the decoder chooses from 
among them arbitrarily. 

4) Round Conclusion: At the end of round /, the decoder 
produces y(I + 1) by setting 

r(/ + i) = |sGr(/):i(fS(,^(,)))e y (15) 

^ r'eR{S) ^ 

for r/ to be defined such that rj > e and 77 ^ as e ^ 0. 



D. Error Probability 

Define the following error events: 

£2(/)={Jf^1^(/)}, 

£3(1) ^ {<(X{1(^)(/)) 0Q'^_J\ y £i(/,z). 
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The total probability of error is Let 

For any sequence of events Ao,Ai, ■ ■ ■ ,An with Aj c 
and Pr{Ao) = 0, 

That is, ki is the number of subcodebooks that if chosen could 

N pj-zy^c-j ^ cause an error Recall that sensor i chooses the subcodebook 

Pr(yijv) = 1 — Y\. p /yrc^ ^ ^ ^ n^"*^ ^ randomly from the uniform distribution. Thus, given and 

1=1 ^ ^^1' 1=1 ^^i-i' probability of an error resulting from a bad choice 

^ of subcodebook is fci(a;", i"._^)/C. Furthermore, ki is based 

^ Pr(yi/|yiJ_]^). strictly on the codebook, we can think of fci as a random 

^=1 variable based on the codebook choice. Averaging over all 
Set Ai ^ £2(1 + 1) U U,;ejt U • • • U £i(/,i). This possible codebooks, 
satisfies the conditions, and since X^^(I — 1) — ^ ^(1) 

X^{I) IS a Markov cham, VrtPJT iWPStTW < W \^ ^(^^\ r,.«^ ^'^-^^ 



Pr(^/KJ_i) -Pr(£2(/ + l)U U £i(/,i)|£2(/) 



Therefore 



Pr(£i(/,»)|£^(/))<E^ MxD. max 



where the expectation is taken over all codebooks. 

Let 6 be the set of all codebooks. We define a subset Ci, 
and show that the probability of error can be easily bounded 



/ II t -x CI \\ ™" snow mat tne prooaomty or error can oe easily Dounaea 

^'^ <2^Pr(£2(/+l)U (J £i(/,«) £2U)j- for any codebook in e\ei, and the probability of a codebook 

being chosen in Ci is small. In particular, let Ci be the set of 



1=1 ie'K 



If J{ e and t{XJ^f^,^p)) e Ql^ ,^,, then "K e 1^(1 + 1). codebooks for which, for any x'^ e and xl\_^ e X^._^, 

Thus f^iiK^ ^^si^i) > for an integer i? < C to be defined later. 



£2(1 + l)\£^(/) c {tiXZ^y^il)) ^ QlJ\m) 



Then 



C (£3(1) U y £i(/,i))\£§(/) Pr(£i(/,z)|£§(/)) < Pr(e\ei) ^ p{x^) _^ max^ ^ 

'° +Pr(ei) y pix'l) max § 



^=1 ~ ' <^+Pr(ei). (18) 



TV iV 



<^Pr(£3(/)|£^(/))+5]5]Pr(£i(/,z)|£§(/)). 



Since each subcodebook is generated identically, ki is a 
(25) binomial random variable with C trials and probability of 
success 

We will show that for any /, 

Pr(£3(/)|£^(/)) < ^. (17) P = Pr(3j,xr e T,{xl_J\{xn : F^^cA^T) = F..c,M)) 

<E E Pr(F,,,,(xr) = F,,,,(xn) 

i 2;;"GT,(iJ^_^)\{a;7} 



If the traitors receive perfect source information, then 



£3(/) c ^ rr(x^)} n {xr(/) = xr(/), v^ ^ ,^ , , „„„(,,^.) 

c {x^{i) T,"(xjt)} - r^-^^^.-i^i 

meaning ( flTl l holds for sufficiently large n. Thus ( flTI i is only — n ; — 
nontrivial if the traitors receive imperfect source information. 

This case is dealt with in Section EH f"'" sufficiently large n. For a binomial random variable X 

Now consider Pr(£i(/,z)|£i(/)) for honest Conditioning mean X and any we can use the Chernoff bound to 

on £|(/) ensures that i e 'U(r(/)) for honest i, so (/) ^"^^ 

will be non-null. The only remaining way to make an error ^ , ^ \ ^ ( ^X\ 

Pr(A > Kj < . (19) 



on X" is if there is some transaction j for which there is a \ k ) 

sequence xf E Tj{X^^_J such that 7^ X'^ and Fi^cj has 
the same value for X" and x^". However, Si_i may contain Therefore 
traitors. Indeed, it may be made entirely of traitors. Thus, we 

have to take into account that X" may be chosen to ensure ^ , „ ^„ „x / eCP \ ^ „„ri. ,a 

the existence of such an erroneous . v^w' \B + 1 I ~ 
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if V > e and n is sufficiently large. Thus 

Pr(ei) =Pr(3a;r,£:'^_, : Ux\\xl_^) > B) 

Combining with ^ll}, ([TSll, and (EOll gives 



(20) 



TV 



Pe < - + { — + 2"('°'5|3C,|+log|X,._J+S(£-i/) 

^ /=i ieM 



2 V C* 



which is less than a for sufficiently large n if 

log|XM| 



and 



B > 



3NmB 37Vmlog|XM| 



a(:^ — e) 



E. Code Rate 

The discussion above placed a lower bound on C. However, 
for sufficiently large n, we can make ^ logC < e, meaning it 
takes no more than e rate to transmit the subcodebook index 
c. Therefore the rate for phase i is at most {j + l)e + ly, where 
j is the number of transactions in phase i. Transaction j must 
be the earliest one with xf e Tj{isi_i), otherwise it would 
have been decoded earher. Thus j is the smallest integer for 
which 



meaning 



By O, for all s e r{I + 1), t(£S(r(7))) ^ Ur'eRis) Q'." 
meaning 

e n U Q;V-Q''(^a+l))- (21) 

s£T{I+l) r'eB.{s) 

Combining this with ([TtT i. with probability at least 1 — a, 
tixucr^i))) e Q';}<;^^ n Q''(r(/ + 1)). Therefore with high 
probability the rate for all of round / is at most 

< Htiiucnm) (^u(r)) + ™(2e + i^) 

< sup iJg (Xu(r)) + m(2e + zy) 
geQ;^_,.nQi(-r(/+i)) 

< sup iJg (XU(1^(/+1))) 

gGQ;^ ,,nQ''C:^(/+l)) 

+ log I Xu(r (/))\u(r (7+1)) I + m(2e + i^) 

< sup Hg{Xu(r)) 

-rcjr, geQ5(^nQ"(r) 

+ log |Xu(r(/))\ix(r(/+i)) I + m{2e + v). (22) 



Whenever U{f {I))\U{y {I + 1)) 7^ 0, at least one sensor is 
eliminated. Therefore the second term in (l22l) will be nonzero 
in all but at most m rounds. Moreover, although we have 
needed to bound v from below, we can still choose it such 
that z/ — » as e ^ 0. Thus if N is large enough, the rate 
averaged over all rounds is no more than 

R.i'K.r)^ sup i?,(Xu(r)) + e 

where e as e ^ 0. This is a precisely a-achievable rate 
function. By continuity of entropy, 

limi?,(J{,r) = 

rc.??', geQM,rnQ(-:^) 



sup H,{Xuiiy)) = R*{%,r) 

SO i?*(!K, r) is achievable. 



F. Imperfect Traitor Information 

We now consider the case that the traitors have access 
to imperfect information about the sources. The additional 
required piece of analysis is to prove (flTl l. That is 

Pr(i(^uc^(/))W) ^ Q'kr^i^ = X^\:Ke r(/)) < — . 
We will in fact prove the slightly stronger statement 

Pr(t(X5^nU(r(/)) (-^)^?nU(r(/)) 

(/))^Q?,,J?{er(/)) 

Since we condition on J{ G ^(1), we can assume Jf C 
'U('^(/)). For notational convenience, let Y = X^{I) and 



Z = X. 



■Jnu(r(/)) 



(/), so (|23] | becomes 



2A^' 



Based on their received value of W", the traitors choose a 
value of c and then a series of messages for each traitor in 
The number of messages each traitor actually gets to 
send depends on how long it takes for the decoder to construct 



a source estimate. Let j 



ieTnu{r(i)) 



be a vector 



representing the number of transactions that take place with 
each traitor in U{y{I)). There are Jg- = nieTnu(r(/)) "^^ 
different possible values of j. We can think of any series 
of values of c and messages as a bin (i.e. a subset Z"); 
that is, all sequences that map to the same messages in the 
subcodebooks denoted by the values of c. Let be the 
rate at which the traitors transmit given j. Thus if we let 23^ 
be the set of all bins in the codebook constructed at rate R, 
the traitors are equivalent to a group of potentially random 
functions gj : W" '^R{S}- 

Consider a joint y, z type t. In order for {Y^'z") to have 
type t for a given j, we need > Ht{Z\Y) + v. Thus 

Pr((y"z") e ^tO'Z)) < Pr(3j : > Ht{Z\Y) + 



e.gj(VK")nAJ'(z|y")). 



Let 6 ^ ^, 



St,i = Pr((y", G T^iYW), 

3z" G n AJ'(Z|y")) 
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and 



y = <t : max St > -tt- — — 

\ y.RQ)>H,{Z\Y)+^ " (n + 1) 1^ x ^Ut 

We will show that T C Q'J^ ^, so that 



< Pr(t(r"z") ^ y\3<. e 

<PT{3tey^,j:Rij)>Ht{Z\Y) + iy, 

< Pr((r", wn i t:{jw)) + 



< ,5- 



1 



yxz| 



2^: 



2A^ 



+ l)|»xZ|j^ 

for sufficiently large n. 

Fix t £ y. There is some j with R{j) > Ht{Z\Y) + v 



and (5t j 



> 



-. Any random is a probabilistic 



(n+l)l«xZ|Jr ■ 

combination of a number of deterministic functions, so if this 
lower bound on (5t j holds for a random gj, it must also hold 
for some deterministic gj . Therefore we do not lose generality 
to assume from now on that gj is deterministic. We also drop 
the j subscript for convenience. Our method of proof will be 
to demonstrate that such a functions 5 can only exist if there 
is also a h : "W" Z'"' with almost the same properties. 
That is, if the traitors can fabricate a counterfeit bin made up 
of source sequences, they can fabricate a single counterfeit 
source sequence contained in this bin that works nearly as 
well. 

Define the following sets: 



3z"G5(u;")nAr(Z|y")}, 



2(n + l) 

Applying the definitions of y and St,j gives 
S 

< Pr((y"W^") G T^'iYW) : 3z" e g{W") (1 Al\Z\Y")) 
= p(w")Pr(r" G A';(rlu;")|VK" = w") 

^P-(^"^-^"^^»+ 2(.+ l)l.x.|,, 

meaning Pr(W^" G A^{W)) > ,(„^,)f,,.| . Fix e 
A^(IF). Since C 



2(n + l)|yxz|j^ 



2n{H{Y\W)-e) 



Note also that 



J2 |<?K)nAr(Z|y")| 

y"6T;-(Y|to") 

= J2 |Ar(>"|2") nr;'(r|u;")| 



Setting ^2(2", w") = |AJ'(r|z") nT,"(y|w")| 

(5 



2"eg(u)") 



2(n+ l)|yxz|j^ 



2n{H(Y\W)~ 



(24) 



for sufficiently large n. We will show that there is actually 
a single z" G giw"') such that fc2(i",w") represents a large 
portion of the above sum, so z" itself is almost as good as the 
entire bin. Then setting h{w") — z" will give us the properties 
we need. Note that 

Y Hz\w-)^ J2 \^tiz\yn\ 

<- 2n(H(Y\W)+Ht{Z\Y)+,) _ ^25) 

Certainly 

fc2(z",w") < |T,"(y|w")| < 2"(-f^('*'i'^)+<^) 

so if we let /(z") be the integer such that 

^ 2^(H{Y\W)-(l{z")-l)e) _ (26) 

then /(z") > 0. Furthermore, if fc2(2", w") > 0, then ?(z") < 
L ^ Let M{1) = |{z" G Z" : /(z") /}|. Then 

from d25T l, for some /, 

2niHiYlW)+miZ\Y)+e) ^ ^ fc2(z",u;") 

> M(02"(^(^l^)-'^) 



giving 



M{1) < 2"(^*(^l'^)+('+i)<^). 



For any bin b G S^q), let M{l,b) = |{z" G 6 : Z(z") = 
Since > + ly, M{l,b) is a binomial random 

variable with M{1) trials and probability of success at most 

2-n{H4Z\Y)+u)^ Thus 

EM{l,b) < 2«(^*(^l^)+('+i)')2""(^'(^l'^)+'') 

_ 2n{{l+l)e-v) 

Let 62 be the set of codebooks such that for any group of 
sensors, subcodebooks, type t, transactions j, sequence w" G 
W", bin b and integer I, either M {I, b) > 2"^ if {l + l)e-iy < 
or M{1, b) > 2"(('+2)'=-'') if (l + l)e-iy > 0. We will show 
that the probability of 62 is small, so we may disregard it. 
Again using ([T9]l, if {I + l)e - v < 0, 



Pr(M(/,6) > 2"") < 
and if {I + l)e - > 0, 



2n{-U+i^) 



Pr(M(/,6)>2"«'+2)-'^))< (— 

< 2-2"< 



< 2~ 



^((1 + 2),-..) 
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both for sufficiently large n. Therefore 

Pr(e2) < 2™C™(n + Ji • • ■ J„|W|"2"(I^-'^'I+'^) 



, 0<i<^-l 



which vanishes as n grows. 

We assume from now on that the codebook is not in C2, 
meaning in particular that Af(Z, 5 (?«")) < 2'" for {l+l)e—i' < 
and M{l,g{w")) < 2"(('+2)<^-'^) for {I + l)e - u > 0. 
Applying these and (|26] | to (l24l i and letting I be an integer 
defined later, 

2-n2e < 2-»^^(^l^) J2 ^2(^",l«") 

z"eg(w'>) 

L 

<^M(/,gK'))2-"('-i)^ 



= ^ Af(/,(7(w"))2-"('-i)'^ 

0</<f 

+ ^ A/(/,5(u;"))2-"('-i)^ 
z<;<f -1 

+ 5] MG,5(u;"))2-"('-i)^ 
f -i<;<L 

< J2 M(^ff(w"))2"' + ^ 2"'^2-"('-i)<= 

0<l<l '<'<7-l 

^-i<;<L 

< ^ M(/,g(w"))2"'+L2"(-'+2)£_^2.2"(3^-''). 



o</<; 

Therefore 

M{l,g{w")) > 2-"^' (1 - L2"(-'+*)' - ^2''{5e-^) 

0<l<l 

Setting / — 5 and ly > 5e ensures that the right hand side 
is positive for sufficiently large n, so there is at least one 

z" G g{w") with |r,"(y|w") n Aj'(yiz")| > 2"(-«"('*'i^)-4^). 

Now we define h : W" — * Z" such that h{w'^) is such a z" 
for e A^{W) and is arbitrary for w" ^ A^(VK). 

If we let Z" = /i(VK"), 

Pr((r"z") e Ar(i^^)) 

> ^ Pr(r" e AJ'(r|/i(u;"))|VK" = w") 

• Pr (y " e (r I ) n AJ' (F 1 ) ) 1 H/" = ) 

> Pr(W^" e ^n(-yp.))2-"(^(^l^)+')2"(^(^l"')-'*') 
(5 



> 



2(n + l)|yxZ| 
The variables {Y^W"Z") are distributed according to 



Let qi{ywz) be the marginal distribution of q'^{y'^w'^ z"^) at 
time i. It factors as 

qi{ywz) p{y)r{w\y)qi{z\w). 

Let q{yz) = ^Ej9»(2/^) and = ^ ELi 

Then 

SO by Lemma I2] 



D t 



Piy)Yr{w\y)q{z\w) 



< log 



5e. 



Therefore t e Q!J^ for sufficiently large n and some t] such 
that 77 ^ as e ^ 0. 

G. Eavesdropping Traitors 

We consider now the case that the traitors are able to 
overhear communication between the honest sensors and the 
decoder If the traitors have perfect information, then hearing 
the messages sent by honest sensors will not give them any 
additional information, so the above coding scheme still works 
identically. If the traitors have imperfect information, we need 
to slightly modify the coding scheme, but the achievable rates 
are the same. 

The important observation is that eavesdropping traitors 
only have access to messages sent in the past. Thus, by 
permuting the order in which sensors are polled in each 
round, the effect of the eavesdropping can be eliminated. 
In a given round, let 5f' be the set of honest sensors that 
transmit before any traitor Since the additional information 
gain from eavesdropping will be no more than the values of 
A"^,, the rate for this round, if no sensors are eliminated 
(i.e. U{r{I + 1)) = U(r(/))), will be no more than the 
rate without eavesdropping when the traitors have access to 
W" = {W^ , XJ^,). The goal of permuting the transmission 
order is to find an ordering in which all the traitors transmit 
before any of the honest sensors, since then the achieved rate, 
if no sensors are eliminated, will be the same as with no 
eavesdropping. It is possible to determine when such an order 
occurs because it will be the order that produces the smallest 
rate. 

More specifically, we will alter the transmission order from 
round to round in the following way. We always choose an 
ordering such that for some § € the sensors §^ transmit 
before §. We cycle through all such orderings until for each 
§, there has been one round with a corresponding ordering in 
which no sensors were eliminated. We then choose one S that 
never produced a rate larger than the smallest rate encountered 
so far. We perform rounds in a order corresponding to S from 
then on. If the rate ever changes and is no longer the minimum 
rate encountered so far, we choose a different minimizing 
8. The minimum rate will always be no greater than the 
achievable rate without eavesdropping, so after enough rounds, 
we achieve the same average rate. 
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VI. Fixed-Rate Coding 

Consider an m-tuple of rates (i?i, • ■ • , Rm), encoding func- 
tions /, : Xf {!,■■■ ,2"^'} for i £ M, and decoding 
function 



g:n{lr--,2"«-}->X5'x 



X X™ 



1=1 



Let li G {1, • • • , 2"^'} be the message transmitted by sensor 
i. If sensor i is honest, = fi{Xf). If it is a traitor, it may 
choose li arbitrarily, based on W^. Define the probability of 
eii-or P, ^ Pr {X-^ ^ XJ^) where X^ = g{h, • • • , 

We say an m-tuple • • • , Rm) is deterministic-fixed-mte 
achievable if for any e > and sufficiently large n, there 
exist coding functions fi and g such that, for any choice of 
actions by the traitors, Pg < e. Let 3?(jfi- C M™ be the set of 
deterministic-fixed-rate achievable m-tuples. 

For randomized fixed-rate coding, the encoding functions 
become 

/, :XrxZ^{l,---,2"^'} 

where Z is the alphabet for the randomness. If sensor i is 
honest, Ii = fi{Xl^,pi), where pi G Z is the randomness 
produced at sensor i. Define an m-tuple to be randomize d- 
fixed-rate achievable in the same way as above, and %ir C M™ 
to be the set of randomized-fixed-rate achievable rate vectors. 

For any § C M, let SW(Xs) be the Slepian-Wolf rate region 
on the random variables X$. That is, 

SW(Xs) = |i?s : V§' C § : i?, > i?(XsHXs\S')|- 



Let 



= {{Ri, • • • , i?™) : V§ e : i?s e SW(Xs)}, 

3^dfr - {(^1, ■■■ ,Rm)& 3?* : V§i, §2 e : 

if 3r e R{§2) ■■ HriXs.nS-M) = 0, 
then i?SinS2 e SW(XsinS2)} 

The following theorem gives the rate regions explicitly. 
Theorem 2: The fixed-rate achievable regions are given by 



and 



3^rfr — S^ifr- 



VII. Proof of Theorem|2] 

A. Converse for Randomized Coding 

Assume (i?i, • • • , Rm) is randomized-fixed-rate achievable. 
Fix S G Jf. Suppose S"^ are the traitors and perform a black 
hole attack. Thus Xg must be based entirely on {fi{X'^)}i^$, 
and since Pr(Xs ^ Xg) can be made arbitrarily small, by the 
converse of the Slepian-Wolf theorem, which holds even if the 
encoders may use randomness, R$ G SW(Xs). 

B. Converse for Deterministic Coding 

Assume ,Rm) is deterministic-fixed-rate achiev- 

able. The converse for randomized coding holds equally well 
here, so {Ri,--- ,Rm) G 3^ifi-- We prove by contradiction 
that (i?i, • • • , Rm) e 3?(5fr well. Suppose (i?i, • • ■ , Rm) G 
3l*fY\3l(Jfj., meaning that for some §i,§2 G J^, there exists 



r G i?(S2) such that Hr{Xs,n$2\W) - but Rs,nS2 ^ 
SW(XsinS2)- Consider the case that "K — and r is such 
that Hr{§i n3i|VF) — 0. Thus the traitors always have access 

For all § e Jf, let D{Xs) be the subset of r,"(Xs) 
such that all sequences in D are decoded correctly if are 
the traitors and no matter what messages they send. Thus 
the probabiUty that G D{Xs) is large. Let D{Xsj^n:K) 
be the marginal intersection of D{X$-^) and D{Xj^). That 
is, it is the set of sequences Xg^pj^- such that there ex- 
ists and x'^^g^ with (xg^nM^^SAM) ^ D{X§,) and 
(•^s nM^MXSi) G D{Xj{). Note that with high probability 
^SinM e D{Xs,n^). Suppose X^^nM ^ ■D(Xs,nM) and 
(X^^nM^M\Si) ^ ■D(Xm), so by the definition of D, 
^SinM = ^s\nw Since PsinM ^ SW(Xs,nJf), there is 
some x'g"pj^ G £'(XsinM) mapping to the same codewords 
as Xs-^nK such that x'^^^^^ ^ X^^^^. Because the traitors 
have access to XsjnM, they can construct Xg^^r^, and also 
find Xg"^^^ such that (a^s" nM-^slvjc) ^ -D(^Si). If the traitors 
report x'g" ^j^, then we have a contradiction, since this situation 
is identical to that of the traitors being S^j;, in which case, by 



the definition of _D, X 



C. Achievability for Deterministic Coding 

Fix (i?!,-- - e S^dfr. Our achievability scheme will 

be a simple extension of the random binning proof of the 
Slepian-Wolf theorem given in [14]. Each encoding function 

: Xf {1, • • • , 2"^'} is constructed by means of a random 
binning procedure. Decoding is then performed as follows. For 
each § G Jf, if there is at least one Xg G T"(Xs) matching 
all received codewords from §, let a;"g be one such sequence 
for all i G s. If there is no such sequence, leave i"g null. 
Note that we produce a separate estimate a;"g of X" for all 
§ 9 i. Let i" equal one non-null i"g. 

We now consider the probability of error. With high prob- 
ability, — X" for honest i. Thus all we need to show 
is that for all other § G with j G §, Xi^$ is null or also 
equal to X". Fix § G If there is some r G R{§) with 
Hr{Xjins\W) = 0, then by the definition of 3?Jf^, P^ns G 
SW(Xjfns)- Thus with high probability the only sequence 
x^pg G T"(Xj£ns) matching all received codewords will be 
X;^pg, so i^g = XI' for alH G 5{ n S. 

Now consider the case that Hr{Xj^r]$\W) > for all 
r G i?(S). For convenience, let Y — Xj^ns and Z = X^. 
Let Ry = Liejfns-Ri Rz = Y.^^j Rt- Since i?s G 
SW(Xs), Ry + Rz > H{YZ) + T] for some Let 
be the set of sequences in y" that map to the same codewords 
as y", and let hz C Z" be the set of sequences mapping to 
the codewords sent by the traitors. Then Y may be decoded 
incorrectly only if there is some y'" G 6y(y") and some 
G bz such that y'" ^ Y" and G T^{YZ). For 

some G W", 

Pr(V" e fey(r")\{r"},z" g bz 

G T^'{YZ)\W" = w") 

<Pr(r"^r,"(y|w")|M^" = w")+ ^ p(y"lu'") 

j/"GTj'(y|ui") 
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<e + 2-«W^I^)-^) ^ fc3(z",z«") (27) 

z"GbznTp(Z) 

where 

V"e^^y(y")nT:(y|z")\{y"}}|. 

On average, the number of typical y" put into a bin is at 
most 2"(^(^)^-'^"^+'^), so we can use ( fT9] l to assume with high 
probability than no more than 2"(^(^)^^^+^^) are put into 
any bin. Note that 

< E E |V(2/")n7;"(r|z")\{y"}| 

= E E 

^ 2"(-f^('^|W')+02"(-f^('^)--R->'+202"(-f^"(^l'*')+«) 

_ 2n(H(YZ)+H{Y\W)~RY+4:<i) 

The average ^3 sum over typical z" in a given bin is thus 

2n(H(YZ)+H{Y\W)-RY-Rz+4:<!) ^ 2"(-f^"("^l^)+4«-'7) 

We can use an argument similar to that in Section IV-FI 
partitioning T^{Z) into different / values, to show that with 
high probability, since H{Y\W) > 0, for all bins bz, 

E fc3(z",u;") <2"W^I^)+5^-''). 

z^£Tp{Z)nbz 

Applying this to ( |27] l gives 

Pr(3y'"e6y(r")\{y"},z"e6z: 

Letting 77 > 6e ensures that the probability of error is always 
small no matter what bin bz the traitors choose. 

D. Achievability for Randomized Coding 

We perform essentially the same coding procedure as with 
deterministic coding, expect we also apply randomness in a 
similar fashion as with variable-rate coding. The only differ- 
ence from the deterministic coding scheme is that each sensor 
has a set of C identically created subcodebooks, from which 
it randomly chooses one, then sends the chosen subcodebook 
index along with the codeword. Decoding is the same as 
for deterministic coding. An argument similar to that in 
Section IV-DI can be used to show small probability of error 

VIII. Conclusion 

We gave an explicit characterization of the region of achiev- 
able rates for a Byzantine attack on distributed source coding 
with variable-rate codes, deterministic fixed-rate codes, and 
randomized fixed-rate codes. We saw that a different set of 
rates were achievable for the three cases, and gave converse 



proofs and rate achieving coding schemes for each. Variable- 
rate achievability was shown using an algorithm in which 
sensors use randomness to make it unlikely that the traitors 
can fool the coding process. 

Much more work could be done in the area of Byzantine 
network source coding. Multiterminal rate distortion [15], [16] 
could be studied, or other topologies, such as side information. 
However, perhaps the biggest drawback in this paper is that, as 
we discussed in the introduction, because the traitors cannot in 
general be identified, it is difficult to imagine applications that 
do not require some post processing of the source estimates, 
for example to estimate some underlying process. Thus it 
would make sense to solve the coding and estimation problems 
simultaneously, such as in the the CEO problem [17]. 
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